Text-based or term-based based searching, wherein a user inputs a word or phrase into a search engine and receives a variety of results is a useful tool for searching. Term based queries require a user to explicitly provide search terms in the form of words, phrases and/or other terms. Sometimes a user may see an object in the physical world and want to locate information related to the object, but do know what terms to use for a term-based query in a search engine. Accordingly, a system that can receive a visual query such as an image, and use it to determine one or more terms would be desirable.